Skip to content

Add per_minute, per_hour, per_day function support#4531

Merged
dai-chen merged 7 commits intoopensearch-project:mainfrom
dai-chen:add-per-function-support-minute-hour-day
Oct 24, 2025
Merged

Add per_minute, per_hour, per_day function support#4531
dai-chen merged 7 commits intoopensearch-project:mainfrom
dai-chen:add-per-function-support-minute-hour-day

Conversation

@dai-chen
Copy link
Collaborator

@dai-chen dai-chen commented Oct 13, 2025 •
edited
Loading

Description

This PR extends timechart command's per_* function support (introduced in #4464) by adding three additional per_* functions that normalize metrics to different time units:

  • per_minute(field): Calculates the per-minute rate (sum-per-second multiplies by 60)
  • per_hour(field): Calculates the per-hour rate (sum-per-second multiplies by 3600)
  • per_day(field): Calculates the per-day rate (sum-per-second multiplies by 86400)

TODO: After #4550 is resolved, migrate from the current second-based approach to millisecond-based calculations, e.g., timestampdiff(MILLISECOND, span_start, span_end) to support millisecond span.

Related Issues

Resolves #4350

Check List

  • New functionality includes testing.
  • New functionality has been documented.
  • New functionality has javadoc added.
  • New functionality has a user manual doc added.
  • New PPL command checklist all confirmed.
  • API changes companion pull request created.
  • Commits are signed per the DCO using --signoff or -s.
  • Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@dai-chen
Support per_minute/hour/day function
b3da82b 
Signed-off-by: Chen Dai <daichen@amazon.com>
@dai-chen dai-chen self-assigned this Oct 13, 2025
@dai-chen dai-chen added enhancement New feature or request PPL Piped processing language backport 2.19-dev labels Oct 13, 2025
@dai-chen
Remove unused UT and IT
bed7437 
Signed-off-by: Chen Dai <daichen@amazon.com>
@dai-chen dai-chen marked this pull request as ready for review October 14, 2025 18:01
@dai-chen
Add more test for edge case
b4c9362 
Signed-off-by: Chen Dai <daichen@amazon.com>
RyanL1997
RyanL1997 previously approved these changes Oct 16, 2025
View reviewed changes
Copy link
Collaborator

@RyanL1997 RyanL1997 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @dai-chen , thanks for the change, and it is LGTM. Just left a very minor question on the doc format.

@dai-chen
Address PR comments in doc
28bb1a9 
Signed-off-by: Chen Dai <daichen@amazon.com>
RyanL1997
RyanL1997 previously approved these changes Oct 23, 2025
View reviewed changes
ykmr1224
ykmr1224 previously approved these changes Oct 23, 2025
View reviewed changes
Copy link
Collaborator

@ykmr1224 ykmr1224 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM other than minor comment.

@dai-chen dai-chen dismissed stale reviews from ykmr1224 and RyanL1997 via d12f90d October 24, 2025 15:48
@dai-chen
Copy link
Collaborator Author

dai-chen commented Oct 24, 2025

@RyanL1997 @ykmr1224 Addressed comments and merged from main.

Waiting for CI complete. Please re-approve. Thanks!

@dai-chen dai-chen merged commit 435e4b2 into opensearch-project:main Oct 24, 2025
33 checks passed
opensearch-trigger-bot bot pushed a commit that referenced this pull request Oct 24, 2025
@github-actions
Add per_minute, per_hour, per_day function support (#4531)
d22cdcb 
* Support per_minute/hour/day function

Signed-off-by: Chen Dai <daichen@amazon.com>

* Remove unused UT and IT

Signed-off-by: Chen Dai <daichen@amazon.com>

* Add more test for edge case

Signed-off-by: Chen Dai <daichen@amazon.com>

* Address PR comments in doc

Signed-off-by: Chen Dai <daichen@amazon.com>

* Address PR comments in UT

Signed-off-by: Chen Dai <daichen@amazon.com>

---------

Signed-off-by: Chen Dai <daichen@amazon.com>
(cherry picked from commit 435e4b2)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
@opensearch-trigger-bot opensearch-trigger-bot bot mentioned this pull request Oct 24, 2025
Merged
dai-chen pushed a commit that referenced this pull request Oct 24, 2025
@opensearch-trigger-bot @github-actions
Add per_minute, per_hour, per_day function support (#4531) (#4662)
4722ff5 
* Support per_minute/hour/day function



* Remove unused UT and IT



* Add more test for edge case



* Address PR comments in doc



* Address PR comments in UT



---------


(cherry picked from commit 435e4b2)

Signed-off-by: Chen Dai <daichen@amazon.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
asifabashar added a commit to asifabashar/sql that referenced this pull request Oct 28, 2025
@asifabashar
Merge branch 'default-main'
e7775d3 
* default-main: (34 commits)
  Enhance dynamic source clause to support only metadata filters (opensearch-project#4554)
  Make nested alias type support referring to outer context (opensearch-project#4673)
  Update big5 ppl queries and check plans (opensearch-project#4668)
  Support push down sort after limit (opensearch-project#4657)
  Use table scan rowType in filter pushdown could fix rename issue (opensearch-project#4670)
  Fix: Support Alias Fields in MIN, MAX, FIRST, LAST, and TAKE Aggregations (opensearch-project#4621)
  Fix bin nested fields issue (opensearch-project#4606)
  Add `per_minute`, `per_hour`, `per_day` function support (opensearch-project#4531)
  Pushdown sort aggregate metrics (opensearch-project#4603)
  Followup: Change ComparableLinkedHashMap to compare Key than Value (opensearch-project#4648)
  Mitigate the CI failure caused by 500 Internal Server Error (opensearch-project#4646)
  Allow renaming group-by fields to existing field names (opensearch-project#4586)
  Publish internal modules separately for downstream reuse (opensearch-project#4484)
  Revert "Update grammar files and developer guide (opensearch-project#4301)" (opensearch-project#4643)
  Support Automatic Type Conversion for REX/SPATH/PARSE Command Extractions (opensearch-project#4599)
  Replace all dots in fields of table scan's PhysType (opensearch-project#4633)
  Return comparable LinkedHashMap in `valueForCalcite()` of ExprTupleValue (opensearch-project#4629)
  Refactor JsonExtractAllFunctionIT and MapConcatFunctionIT (opensearch-project#4623)
  Pushdown case function in aggregations as range queries (opensearch-project#4400)
  Update GEOIP function to support IP types as input (opensearch-project#4613)
  ...

# Conflicts:
#	docs/user/ppl/functions/conversion.rst
asifabashar added a commit to asifabashar/sql that referenced this pull request Oct 28, 2025
@asifabashar
Merge branch 'default-main'
42d705a 
* default-main: (34 commits)
  Enhance dynamic source clause to support only metadata filters (opensearch-project#4554)
  Make nested alias type support referring to outer context (opensearch-project#4673)
  Update big5 ppl queries and check plans (opensearch-project#4668)
  Support push down sort after limit (opensearch-project#4657)
  Use table scan rowType in filter pushdown could fix rename issue (opensearch-project#4670)
  Fix: Support Alias Fields in MIN, MAX, FIRST, LAST, and TAKE Aggregations (opensearch-project#4621)
  Fix bin nested fields issue (opensearch-project#4606)
  Add `per_minute`, `per_hour`, `per_day` function support (opensearch-project#4531)
  Pushdown sort aggregate metrics (opensearch-project#4603)
  Followup: Change ComparableLinkedHashMap to compare Key than Value (opensearch-project#4648)
  Mitigate the CI failure caused by 500 Internal Server Error (opensearch-project#4646)
  Allow renaming group-by fields to existing field names (opensearch-project#4586)
  Publish internal modules separately for downstream reuse (opensearch-project#4484)
  Revert "Update grammar files and developer guide (opensearch-project#4301)" (opensearch-project#4643)
  Support Automatic Type Conversion for REX/SPATH/PARSE Command Extractions (opensearch-project#4599)
  Replace all dots in fields of table scan's PhysType (opensearch-project#4633)
  Return comparable LinkedHashMap in `valueForCalcite()` of ExprTupleValue (opensearch-project#4629)
  Refactor JsonExtractAllFunctionIT and MapConcatFunctionIT (opensearch-project#4623)
  Pushdown case function in aggregations as range queries (opensearch-project#4400)
  Update GEOIP function to support IP types as input (opensearch-project#4613)
  ...

Signed-off-by: Asif Bashar <asif.bashar@gmail.com>
expani pushed a commit to vinaykpud/sql that referenced this pull request Nov 4, 2025
@dai-chen @expani
Add per_minute, per_hour, per_day function support (opensearch-…
025b6c3 
…project#4531)

* Support per_minute/hour/day function

Signed-off-by: Chen Dai <daichen@amazon.com>

* Remove unused UT and IT

Signed-off-by: Chen Dai <daichen@amazon.com>

* Add more test for edge case

Signed-off-by: Chen Dai <daichen@amazon.com>

* Address PR comments in doc

Signed-off-by: Chen Dai <daichen@amazon.com>

* Address PR comments in UT

Signed-off-by: Chen Dai <daichen@amazon.com>

---------

Signed-off-by: Chen Dai <daichen@amazon.com>
This was referenced Jan 11, 2026
Merged
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ykmr1224 ykmr1224 ykmr1224 approved these changes

@RyanL1997 RyanL1997 RyanL1997 approved these changes

@ps48 ps48 Awaiting requested review from ps48 ps48 is a code owner

@kavithacm kavithacm Awaiting requested review from kavithacm kavithacm is a code owner

@derek-ho derek-ho Awaiting requested review from derek-ho derek-ho is a code owner

@joshuali925 joshuali925 Awaiting requested review from joshuali925 joshuali925 is a code owner

@YANG-DB YANG-DB Awaiting requested review from YANG-DB YANG-DB is a code owner

@mengweieric mengweieric Awaiting requested review from mengweieric mengweieric is a code owner

@vamsimanohar vamsimanohar Awaiting requested review from vamsimanohar vamsimanohar is a code owner

@Swiddis Swiddis Awaiting requested review from Swiddis Swiddis is a code owner

@penghuo penghuo Awaiting requested review from penghuo penghuo is a code owner

@seankao-az seankao-az Awaiting requested review from seankao-az seankao-az is a code owner

@MaxKsyunz MaxKsyunz Awaiting requested review from MaxKsyunz MaxKsyunz is a code owner

@Yury-Fridlyand Yury-Fridlyand Awaiting requested review from Yury-Fridlyand Yury-Fridlyand is a code owner

@anirudha anirudha Awaiting requested review from anirudha anirudha is a code owner

@forestmvey forestmvey Awaiting requested review from forestmvey forestmvey is a code owner

@acarbonetto acarbonetto Awaiting requested review from acarbonetto acarbonetto is a code owner

@GumpacG GumpacG Awaiting requested review from GumpacG GumpacG is a code owner

@LantaoJin LantaoJin Awaiting requested review from LantaoJin LantaoJin is a code owner

@noCharger noCharger Awaiting requested review from noCharger noCharger is a code owner

@qianheng-aws qianheng-aws Awaiting requested review from qianheng-aws qianheng-aws is a code owner

@yuancu yuancu Awaiting requested review from yuancu yuancu is a code owner

Assignees

@dai-chen dai-chen

Labels

backport 2.19-dev enhancement New feature or request PPL Piped processing language

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[FEATURE] PPL per_* aggregation function support

3 participants

@dai-chen @ykmr1224 @RyanL1997